Overview of the Full-Text Document Retrieval Benchmark

8.1 Introduction For most of recorded history, textual data have existed primarily in hardcopy format, and the related document retrieval process was essentially a manual task, possibly involving the assistance of cross-reference catalogs. By the mid-1960s, work was under way at the University of Pittsburgh to develop computer-assisted legal research systems [Harrington, 1984–85]. Also, during this period of time, computer-based document retrieval systems were beginning to emerge in commercial firms; for example, InfoBank at the New York Times [Harrington, 1984–85]. The most distinguishing characteristics of such systems include full-text Boolean search logic and support for proximity expressions (e.g., phrases). With this technology, termed full-text retrieval (FTR), documents are selected from a database in terms of content, rather than with predefined keywords or subject categories. For example, suppose that we were interested in locating articles about benchmarking full-text document retrieval systems. To formulate a search expression that would specify the desired content, we could select keywords (e.g., benchmark, performance) and phrases (e.g., document retrieval, full-text retrieval, information retrieval) which would likely be found within relevant documents. The reader should note that this simple example illustrates one important shortcoming of FTR systems: The inherent ambiguity of natural language makes FTR query formulation imprecise. Although FTR systems lack closed methods to formulate